<html>
<head><meta charset="utf-8"><title>jobserver passthrough in build.rs · general · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/index.html">general</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html">jobserver passthrough in build.rs</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="211493058"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211493058" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211493058">(Sep 28 2020 at 13:38)</a>:</h4>
<p>The <a href="https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-build-scripts">docs</a> say that doing <code>make -j $NUM_JOBS</code> from <a href="http://build.rs">build.rs</a> is not the way to enable multiple jobs for make, but rather jobservers should be used.</p>
<p>But then what is the way to pass through jobserver support in <a href="http://build.rs">build.rs</a>? <a href="https://github.com/randombit/botan-rs/blob/5c83aa56b346557e480eceb5b652cd08ac95e96e/botan-src/src/lib.rs#L119">This</a> is the current code, and it's not parallel.</p>



<a name="211493172"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211493172" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211493172">(Sep 28 2020 at 13:39)</a>:</h4>
<p>Cargo itself has a jobserver builtin. It passes it through to at least all rustc invocations. Maybe it doesn't to <a href="http://build.rs">build.rs</a> invocations?</p>



<a name="211493368"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211493368" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211493368">(Sep 28 2020 at 13:41)</a>:</h4>
<p>It should be passed through: <a href="https://github.com/rust-lang/cargo/blob/cb7cff15c8b0a78fccdc5d797485490dc3bd6c85/src/cargo/core/compiler/custom_build.rs#L198">https://github.com/rust-lang/cargo/blob/cb7cff15c8b0a78fccdc5d797485490dc3bd6c85/src/cargo/core/compiler/custom_build.rs#L198</a></p>



<a name="211493849"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211493849" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211493849">(Sep 28 2020 at 13:45)</a>:</h4>
<p>That's weird... the code is in a <a href="http://lib.rs">lib.rs</a> but it's <a href="https://github.com/randombit/botan-rs/blob/5c83aa56b346557e480eceb5b652cd08ac95e96e/botan-sys/build.rs#L4">invoked through a build.rs script</a>.</p>



<a name="211494073"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211494073" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211494073">(Sep 28 2020 at 13:47)</a>:</h4>
<p>Maybe the <a href="http://build.rs">build.rs</a> can only act as a client but doesn't pass through the information to subprocesses?</p>



<a name="211494244"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211494244" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211494244">(Sep 28 2020 at 13:49)</a>:</h4>
<p>I think it should either just work (due to the environment the build script is run in), or you can perhaps use the jobserver crate and Client::from_env and then configure_cmd on the Make invocation</p>



<a name="211494873"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211494873" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211494873">(Sep 28 2020 at 13:54)</a>:</h4>
<p>maybe process::Command doesn't inherit fd's?</p>



<a name="211495007"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211495007" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211495007">(Sep 28 2020 at 13:55)</a>:</h4>
<p>apparently there is a MAKEFLAGS env var which contains a string that has the fd paths needed for communication</p>



<a name="211495015"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211495015" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211495015">(Sep 28 2020 at 13:55)</a>:</h4>
<p><a href="https://docs.rs/jobserver/0.1.21/src/jobserver/unix.rs.html#66">https://docs.rs/jobserver/0.1.21/src/jobserver/unix.rs.html#66</a></p>



<a name="211496758"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496758" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496758">(Sep 28 2020 at 14:06)</a>:</h4>
<p><span class="user-mention" data-user-id="120096">@est31</span> <br>
Do you run cargo from an outer make-based build system?</p>



<a name="211496773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496773">(Sep 28 2020 at 14:07)</a>:</h4>
<p>No, directly</p>



<a name="211496831"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496831" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496831">(Sep 28 2020 at 14:07)</a>:</h4>
<p>Then from what do you want to inherit jobserver?</p>



<a name="211496852"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496852" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496852">(Sep 28 2020 at 14:07)</a>:</h4>
<p>cargo</p>



<a name="211496940"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496940" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496940">(Sep 28 2020 at 14:08)</a>:</h4>
<p>I think I'll have to do what <span class="user-mention" data-user-id="116122">@simulacrum</span>  suggests</p>



<a name="211496945"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496945" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496945">(Sep 28 2020 at 14:08)</a>:</h4>
<p>Ah, ok, cargo <a href="http://build.rs">build.rs</a> runs something and that something needs to inherit jobserver from cargo.</p>



<a name="211496976"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211496976" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211496976">(Sep 28 2020 at 14:08)</a>:</h4>
<p><a href="https://docs.rs/jobserver/0.1.21/jobserver/struct.Client.html#return-value">https://docs.rs/jobserver/0.1.21/jobserver/struct.Client.html#return-value</a></p>
<blockquote>
<p>Additionally on Unix this function will configure the file descriptors with CLOEXEC so they're not automatically inherited by spawned children.</p>
</blockquote>



<a name="211497095"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211497095" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Vadim Petrochenkov <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211497095">(Sep 28 2020 at 14:09)</a>:</h4>
<p>It's indeed usually done through MAKEFLAGS, but you have to make sure that the descriptors passed through MAKEFLAGS are not closed before the sub-command is run (e.g. make closes them by default, not sure about cargo).</p>



<a name="211497223"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211497223" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211497223">(Sep 28 2020 at 14:10)</a>:</h4>
<p>Cargo uses <code>c.configure()</code> which disables <code>CLOEXEC</code>.</p>



<a name="211497242"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211497242" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211497242">(Sep 28 2020 at 14:11)</a>:</h4>
<p><a href="https://docs.rs/jobserver/0.1.21/src/jobserver/unix.rs.html#191">https://docs.rs/jobserver/0.1.21/src/jobserver/unix.rs.html#191</a></p>



<a name="211497297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211497297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211497297">(Sep 28 2020 at 14:11)</a>:</h4>
<p>Yeah that's the problem... I think docs should be improved</p>



<a name="211500126"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211500126" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211500126">(Sep 28 2020 at 14:32)</a>:</h4>
<p>.... now I'm trying to do it and get a "bad file descriptor" error...</p>



<a name="211503208"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211503208" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211503208">(Sep 28 2020 at 14:53)</a>:</h4>
<p>This is how it's used in the wild... makes me scratch my head a bit tbh, because it doesnt really seem to be used in the way it's supposed to be <a href="https://github.com/twistedfall/opencv-rust/blob/master/build.rs#L106">https://github.com/twistedfall/opencv-rust/blob/master/build.rs#L106</a></p>



<a name="211503269"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211503269" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211503269">(Sep 28 2020 at 14:54)</a>:</h4>
<p>At that point why do you need the jobserver crate at all...</p>



<a name="211503484"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211503484" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211503484">(Sep 28 2020 at 14:56)</a>:</h4>
<p>I guess that from_env + configure doesn't really work :/</p>



<a name="211506787"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211506787" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211506787">(Sep 28 2020 at 15:19)</a>:</h4>
<p>So I found the issue I think</p>



<a name="211506806"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211506806" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211506806">(Sep 28 2020 at 15:19)</a>:</h4>
<p><a href="https://github.com/alexcrichton/jobserver-rs/issues/24">https://github.com/alexcrichton/jobserver-rs/issues/24</a></p>



<a name="211506847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211506847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211506847">(Sep 28 2020 at 15:19)</a>:</h4>
<p>If I patch the jobserver crate to also pass <code>cmd.env("MAKEFLAGS", &amp;value);</code></p>



<a name="211506941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211506941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211506941">(Sep 28 2020 at 15:20)</a>:</h4>
<p>then it works and Command::new("make") automatically uses the jobserver implementation</p>



<a name="211506961"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211506961" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211506961">(Sep 28 2020 at 15:20)</a>:</h4>
<p>No need to use the jobserver crate or anything</p>



<a name="211507905"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211507905" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211507905">(Sep 28 2020 at 15:27)</a>:</h4>
<p><a href="https://github.com/alexcrichton/jobserver-rs/issues/24#issuecomment-700083933">https://github.com/alexcrichton/jobserver-rs/issues/24#issuecomment-700083933</a></p>



<a name="211507978"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211507978" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211507978">(Sep 28 2020 at 15:28)</a>:</h4>
<p>Yeah I think that issue should be resolved, and cargo should set MAKEFLAGS</p>



<a name="211508060"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211508060" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211508060">(Sep 28 2020 at 15:29)</a>:</h4>
<p>(regarding the patching above, I mean patching cargo to also pass MAKEFLAGS)</p>



<a name="211551893"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/122651-general/topic/jobserver%20passthrough%20in%20build.rs/near/211551893" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> est31 <a href="https://rust-lang.github.io/zulip_archive/stream/122651-general/topic/jobserver.20passthrough.20in.20build.2Ers.html#211551893">(Sep 28 2020 at 21:19)</a>:</h4>
<p>There we go: <a href="https://github.com/rust-lang/cargo/pull/8741">https://github.com/rust-lang/cargo/pull/8741</a></p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>